Papers with realistic scenario
MUG: Interactive Multimodal Grounding on User Interfaces (2024.findings-eacl)
Copied to clipboard
| Challenge: | Prior studies modeled multimodal UI grounding in one round, but such an interaction is inherently iterative. |
| Approach: | They propose a task where a user and an agent collaborate on an interface screen . they use a dataset of 77,820 sequences of human user-agent interaction on mobile interfaces . |
| Outcome: | The proposed task improves the absolute task completion by 18% over the entire test set and 31% over the challenging split. |
Exploring Backdoor Vulnerabilities of Chat Models (2025.coling-main)
Copied to clipboard
| Challenge: | Recent studies show that Large Language Models (LLMs) are susceptible to a security threat known as Backdoor Attack. |
| Approach: | They propose a backdoor attack method that distributes trigger scenarios across user inputs in different rounds and makes the backdoor be triggered only when all trigger scenarios have appeared in the historical conversations. |
| Outcome: | The proposed method achieves high attack success rates on chat models while maintaining normal capabilities on providing helpful responses to benign user requests. |
Cross-Domain Detection of GPT-2-Generated Technical Text (2022.naacl-main)
Copied to clipboard
| Challenge: | Recent advances in techniques for generating realistic synthetic content pose a diverse set of problems with significant societal consequences. |
| Approach: | They propose to use paragraph-level detectors to detect tampering of full-length documents under a variety of threat models to detect machine-generated text. |
| Outcome: | The proposed detectors can detect the tampering of full-length documents under a variety of threat models. |
ProtoInfoMax: Prototypical Networks with Mutual Information Maximization for Out-of-Domain Detection (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods to detect out-of-domain (OOD) inputs are limited and lack data. |
| Approach: | They propose a new architecture that extends Prototypical Networks to process in-domain and OOD sentences via Mutual Information Maximization objective. |
| Outcome: | The proposed method significantly improves performance up to 20% for OOD detection in low resource settings of text classification. |
Complex Claim Verification with Evidence Retrieved in the Wild (2024.naacl-long)
Copied to clipboard
| Challenge: | Prior work makes simplifying assumptions in retrieval that depart from real-world use cases: no access to evidence, access to curated evidence, or access to published evidence after a claim was made. |
| Approach: | They propose a pipeline to check claims using raw evidence from the web . they restrict their retriever to only search documents available prior to the claim's making . |
| Outcome: | The proposed method is based on a political claim dataset and shows that the evidence summary produced by the system is reliable and relevant to answering key questions. |